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Introduction and Problem Statement 


The modus operand i in addressing human error in aviation systems is 
predominantly that of technological interventions or “fixes.” Such interventions exhibit 
considerable variability both in terms of sophistication and application. Some 
technological interventions address human error directly while others do so only 
indirectly. Some attempt to eliminate the occurrence of errors altogether whereas others 
look to reduce the negative consequences of these errors. In any case, technological 
interventions add to the complexity of the systems and may interact with other system 
components in unforeseeable ways and often create opportunities for novel human errors. 
Consequently, there is a need to develop standards for evaluating the potential safety 
benefit of each of these intervention products so that resources can be effectively invested 
to produce the biggest benefit to flight safety as well as to mitigate any adverse 
ramifications. The purpose of this project was to help define the relationship between 
human error and technological interventions, with the ultimate goal of developing a set of 
standards for evaluating or measuring the potential benefits of new human error “fixes.” 

It must be acknowledged and understood, however, how difficult such an endeavor is, 
given the wide variety of technologies and equally varied circumstances for their use, as 
well as the abundance of human factors data that currently exists in the literature. 
Therefore, we approached the problem by developing a framework for summarizing the 
overabundance of data in a manner that best suits two specific types of questions that 
users might have: 

1 . Given a new technology is to be implemented, which specific types or classes of 
human error will most likely be affected, and 

2. Given that a certain type or class of human error has been identified as a major 
safety problem (e.g., decision errors), what kinds of technologies will most likely 
help in alleviating the problem? 

Obviously, each question has a different emphasis and each serves a different 
function. Therefore, the human error data will need to be organized in a manner that 
allows both types of questions to be answered. In essence, the human error classes will 
need to be mapped onto the different classes of technology and vice versa. However, the 
difficulties in associated with a simple matrix are considerable for several reasons. First, 
there is no general consensus in the literature about the terminology used to categorize 
and classify errors (Senders & Moray 1991). Several different taxonomies of human 
error exist with varying degrees of overlap. Furthermore, there is currently no generally 
agreed-upon framework for classifying different technologies or intervention strategies 
(Wiegmann & Shappell, 1997). Thus, interconnecting human error classes with 
intervention technologies in a manner that is both meaningful and useful is indeed 
challenging (Reason, 1 990). 

The purpose of the present project was to explore the possibility of developing a 
comprehensive error-technology matrix for mapping error categories onto technology 
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fixes and vice versa. The project had two distinct phases. The first phase, representing the 
first year’s efforts, involved a review of the repositories of human factors data in order to 
examine the nature and complexity of data sources, terminologies, and classifications 
used to relate human error with technology intervention strategies (see Wiegmann & 
Rantanen, 2002 for summary of these efforts). This information was then used in Year 2 
to map NASA’s Aviation Safety Program (AvSP) interventions onto human error classes 
and to develop a prototype database that provides a structure for linking errors and 
technologies to help evaluate the impact that safety products will have on each error 
class. 


Summary of Year 2 Efforts 

Year 2 efforts focused on evaluating the 48 technology interventions previously 
generated by AvSP. Each of these is listed in Appendix A. Our evaluation consisted of 
several steps. The first step was to analyze each intervention based on its underlying 
nature or function as an intervention. This analysis involved clustering interventions into 
groups or categories using various taxonomies of intervention strategies. The goal of this 
effort was to determine the general focus or distribution of interventions. The second step 
involved mapping these interventions onto error categories to determine the extent to 
which the interventions addressed various classes of human error across different stages 
of accident causation (e.g., pre-crash, crash, and post-crash events). Finally, the third 
phased focus on the issue of evaluating the magnitude of impact that each intervention 
might have on the actual error category it targeted. 

Intervention Categories 

The AvSP technologies were initially classified by NASA into seven broad 
categories. These categories are accident mitigation, aviation system modeling and 
monitoring system, single aircraft accident prevention, synthetic vision systems, system- 
wide accident prevention, weather accident prevention, and aircraft icing. However, such 
categorization focuses more on the surface structure of the technologies rather than on 
their underlying theoretical or functional purpose (or at a minimum reflects a 
combination thereof). Therefore, we applied more traditional intervention taxonomies to 
better understand the nature of each intervention in an attempt to facilitate the mapping of 
these technologies onto error classes. 

Historically, researchers and practitioners alike have proposed various ways of 
viewing safety interventions. In the following sections, we will briefly review each of 
these approaches and discuss how the AvSP technologies might be viewed using each 
approach. These approaches are hazard-centered, function-centered, mode-centered, and 
epidemiological approaches. Each is addressed in turn in the following sections. 

Hazard-Centered Approach 

Hazard-centered interventions focus specifically on addressing the threat to 
operational safety posed by a particular situation or event. As described by Wood (1979) 
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and recommended by the International Civil Aviation Organization (ICAO, 1993), 
hazard-centered interventions can be classified into one of three levels: 

1 . Level one safety actions completely remove the offending safety hazard 

2. Level two safety actions modify the system so as to reduce the risk of the 
underlying hazard; and 

3. Level three safety actions accept that the hazard can be neither eliminated nor 
reduced (controlled), and therefore aims at teaching people how to cope with it. 

Figure 1 shows the results of classifying the AvSP products into the three ICAO 
safety levels. 1 Based on the ICAO classification system, over three-quarters of the 
products aim at modifying the system to reduce the risks of hazards, and much smaller 
percentages teach people how to cope with hazards or are unclassifiable. None of the 
products completely removes a safety hazard. The unclassifiable products were all 
Aviation System Modeling and Monitoring System products that do not directly address 
specific hazards. 



Safety Level 

Figure 1. Percentages of the AvSP products classified into ICAO’s (1993) three safety 
levels. Each intervention was categorized based on its single, most direct application. 

Function-Centered Approach 

Another way of conceptualizing technology interventions is based on whether 
they prevent errors from occurring or mitigate the consequences once an error occurs. 

The former focuses on reducing errors altogether while the latter addresses error recovery 


' Note that Figure 1, as will all other figures presented in this section, are derived from classifying each 
NASA Aviation Safety Product (AvSP) into the single most applicable category for each classification 
system, even if the product addresses more than one category. This simplified the analysis for our initial 
purposes, but some categories may appear not to be as well addressed as they would be if secondary 
classifications were calculated on each product. 


or error management. Figure 2 presents a breakdown of the AvSP interventions when 
examined using these two broad categories error prevention vs. mitigation. 



Prevention Mitigation Prevention/ Unclassifiable 

Mitigation 


Figure 2. Percentages of the AvSP products classified as error prevention or mitigation 
measures. 

As illustrated in Figure 2, nearly half of the products both prevent and mitigate errors, 
31% solely mitigate errors, and 19% solely prevent errors. Product SAAP-6, FAA 
Advisory Circular and Flight Control Systems Verification Methods, accounting for the 
remaining 2%, could not be classified because it does not address error. 

A more elaborate way of classifying intevention fuction has been proposed by 
Maurino, Reason, Johnston and Lee (1995). These functions include: 

1 . To create awareness and understanding of the risks and hazards. 

2. To detect and warn of the presence of off-normal conditions or imminent dangers. 

3. To protect people and the environment from injury and damage. 

4. To recover from off-normal conditions and to restore the system to a safe state. 

5. To contain the accidental release of harmful energy or substances. 

6. To enable the potential victims to escape out-of-control hazards. 

Figure 3 presents the distribution of AvSP technologies using this more elaborate 
function-centered framework. The largest proportion (44%) work primarily to protect 
from hazards, and 33% serve to increase awareness of hazards. The remaining 23% of the 
products work to detect and warn of hazards and to contain hazards. None of the products 
aid recovery and escape as their primary function, though some of the products assume 
them as secondary functions. 
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Awareness Detection/ Protection Recovery Containment Escape 
Warning 

Function 

Figure 3. Percentages of the AvSP products classified into each of Reason’s functions. 
Each product was classified into the single most applicable category. 


Mode-Centered Approach 

Another approach to classifying technology intervetions is based on the mode of 
implimentation or more specifically “what” is changed in the system to improve safety. 
For example, according to Wiegmann and Shappell (2003), intervetions could focus on 
changing the: 

1 . Environment (reduce heat, noise, vibration, etc), 

2. Human (via selection, incentives, training) 

3. Machine (design, strength, capacity) 

4. Task (ordering, timing, automation) 

Using this approach, the primary intervention strategies applied by AvSP products are 
shown in Figure 4. Most of the products increase safety by modifying the machine, or 
hardware components of the aviation system. Much smaller percentages of the products 
modify the human, environment, or task. The two unclassifiable products, Incident 
Reporting Enhancement Tools and Fast-time Simulation of System-wide Risks, either do 
not modify the system in a way that directly impacts the safety of operations or could 
lead to modifications of any of the categories. 
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Intervention Strategy 

Figure 4. Percentages of the AvSP products classified by what they primarily modify to 
reduce accidents. Each product was classified into the single most applicable category. 

Another mode centered approach has been discussed by Maurino et al. (1995). 
According to these authors, safety intervetions can be classified into one of five mode 
categories, including: 

1 . Engineered safety devices (flight management systems, terrain warnings, 
automatic detection and shutdown, etc.). 

2. Policies, standards and controls (administrative and managerial measures 
designed to promote standardized and safe working practices. 

3. Procedures, instructions and supervision (measures aimed at providing local task- 
related know-how). 

4. Training, briefing, drills (the provision and consolidation of technical skills, 
safety awareness and safety knowledge). 

5. Personal protective equipment (anything from safety boots to space suits). 

Based on this classification system, the modes assumed by the AvSP products and 
their proportions are shown in Figure 5. Nearly three-quarters of the products result in 
interventions that primarily take the form of engineered safety features. Nineteen percent 
take the form of standards, policies, and controls, and the remaining ten percent take the 
form of training, briefings, and drills. None of the products primarily takes the form of 
procedures, instructions, and supervision or personal protective equipment, though some 
do operate secondarily as procedures, instructions, and supervision. 
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Engineered Standards, Procedures, Training, Personal 

Safety Features Policies, Instructions, Briefings, Drills Protective 

Controls Supervision Equipment 

Mode 

Figure 5. Percentages of the AvSP products classified into each mode category of 
Reason’s Matrix. Each product was classified into the single most applicable category. 


Epidemiolgical Approach 


The epidemiological approach is based on ideas for preventing illness and injury 
within the occupational and public health arena, such as that presented in Figure 6. As a 
result, approaches to interventions can be classified as primary prevention (reduce the 
risk), secondary intervention (alter ways the system reacts to the risk or hazard) and 
tertiary prevention (minimize the damage due to hazard exposure). 


Occupational 


Individual Stress 


Individual & 

Mental 


Responses/ 



Health 


Early Warning 


Organization 

Risks 


Signs 


Distress 


Primary Prevention 
Aim: To reduce the 
risk factor or change 
the nature of the 
stressor. 


Secondary Prevention 
Aim: To alter the ways 
in which individuals 
respond to the risks 
and stressors. 


Tertiary Prevention 
Aim: To heal those 
who have been traumatized 
or distressed at work. 


Figure 6. Model of occupational health by Quick, Murphy, Hurrrell, and Orman (1993). 
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According to Haddon (1980) when applied to transportation accidents the three types 
of interventions generally map on to pre-accident (primary prevention), accident 
(secondary intervention) and post-accident (tertiary intervention). When AvSP 
interventions are considered using this framework, a distribution such as that presented in 
Figure 7 emerges. A large majority of the AvSP products act to maximize safety 
primarily before an accident or incident takes place. Seventeen percent act primarily 
during an accident or incident to minimize its effects, and one of the products, Elevated 
Flash Point Fuel Technologies, act primarily to minimize post-accident complications 
(e.g., crashworthiness and survivability). 



Sequence 

Figure 7. Percentages of the AvSP products classified by stages of an accident. Each 
product was classified into the single most applicable category. 

Hybrid Approaches 

The process of characterizing safety approaches is somewhat arbitrary and not all 
approaches are mutually exclusive. There can be considerable overlap across approaches. 
Figure 8, for example illustrates how the hazard-centered approach can be linked to the 
mode-centered approach (Diehl, 1 989). Specifically, strategies for eliminating hazards 
are often environment-oriented approaches, where as safety features such as warning 
devices tend to be machine-oriented. Procedural approaches for dealing with hazards, in 
turn, are almost exclusively human-oriented. 
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Figure 8. Accident prevention elements as described by Diehl (1989). 
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Maurino et al. (1995) combined the function- and mode-centered approaches to form 
a hybrid intervention matrix, as shown in Figure 9. When AvSP products are mapped 
onto this matrix, the percentages indicate that nearly 48% of the interventions are 
primarily engineered safety features that protect from hazards or increase awareness of 
hazards. An additional 23% of the products are engineered safety features that aid 
detection and warning of hazards or contain hazards. About 19% of the products focus on 
standards, policies, and controls that address awareness of or protection from hazards, 
and the remaining 1 0% primarily take the form of training, briefings, and drills in 
awareness of, or protection from, hazards. None of the products focus primarily on 
recovery or escape, and none of the products primarily take the form of procedures, 
instructions, and supervision or personal protective equipment. In addition, none of the 
products address detection and warning or containment in any mode other than 
engineered safety features. 
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Figure 9. Maurino et al. (1995) Matrix, indicating the percentages of the AvSP products 
that fall into each cell. Each product was classified into the single most applicable 
category. 
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Haddon (1980) combined the mode-centered and epidemiological approaches to form 
the Haddon Matrix, shown in Figure 10. The percentages inside the matrix indicate the 
primary sequence category and system affected by each AvSP product. As illustrated in 
the figure, the interventions are heavily weighted toward vehicle-specific, pre-event 
factors, with very few focusing on post-event factors and none focusing on human- or 
environment-specific during- or post-event factors. 
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Figure 10. Haddon Matrix, indicating the percentages of the AvSP products that fall into 
each cell. Each product was classified into the single most applicable category. 
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Mapping Interventions onto Human Error Classes 

In order to map intervention strategies onto human error categories, the various types of 
human error need to be identified and defined. In executing this phase of the project, we built 
upon the foundation laid down by the NASA ASAFE program, with regard to the data structures 
and taxonomies already in place. In particular, the Human Factors Analysis and Classification 
System (HFACS) has been selected as the error taxonomy, based on its use by NASA and 
previous reviews of the human-error literature (Wiegmann, Rich, & Shappell, 2000). A 
description of the error categories contained in this framework will be described in the following 
sections. 

The Human Factors Analysis and Classification System 

The Human Factors Analysis and Classification System (see Figure 1 1) is based upon 
Reason’s (1990) model of latent and active failures. It addresses human error at each of four 
levels of failure: 1) unsafe acts of operators (e.g., aircrew), 2) preconditions for unsafe acts, 3) 
unsafe supervision, and 4) organizational influences. The HFACS framework was originally 
developed for the U.S. Navy and Marine Corps as an accident investigation and data analysis 
tool. Since its original development, however, HFACS has been employed by other military 
organizations (e.g., U.S. Army, Air Force, and Canadian Defense Force) as an adjunct to 
preexisting accident investigation and analysis systems. Other organizations such as the FAA 
and NASA have explored the use of HFACS as a complement to preexisting systems within civil 
aviation in an attempt to capitalize on gains realized by the military. These initial attempts, 
performed both at the University of Illinois and the Civil Aerospace Medical Institute (CAMI) 
have been highly successful and have shown that HFACS can be reliably used to analyze the 
underlying human factors causes of both commercial and general aviation accidents (Shappell & 
Wiegmann, 2001 ; Wiegmann & Shappell, 2001). Together, these analyses have helped identify 
general trends in the types of human factors issues and aircrew errors that have contributed to 
both military and civil aviation accidents. 

HFACS: Unsafe Acts 

Since the human error or unsafe acts level within the HFACS framework is central to the 
current research project and program goals, this level of the framework will be described briefly. 
The following sections will then discuss previous research findings and future research 
objectives in detail. 

Defining unsafe acts. Within HFACS, the unsafe acts committed by pilots generally take on 
one of two forms, errors or violations. Errors are generally defined as mental or physical 
activities that fail to achieve their intended outcome. Violations, on the other hand, represent a 
willful disregard for rules and regulations. Some examples of aircrew casual factors associated 
errors and violations can be found in Table 1. 
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Figure 11. Categories of accident causal-factors within HFACS. 


Basic error types. There are essentially three basic error types - skill-based errors, decision 
errors, and perceptual errors. Skill-based behavior is best described as those “stick-and-rudder” 
and other basic flight skills that occur without significant conscious thought. Skill-based errors 
may occur due to individual differences in flying skills or as a result of attention and/or memory 
failures. Attention failures, for example, produce such skill-based errors as a breakdown in visual 
scan patterns, task fixation, the inadvertent activation of controls, or the misordering of steps in a 
procedure. In contrast, memory failures often appear as omitted items in a checklist, place losing, 
or forgotten intentions. Decision errors represent intentional behavior that proceeds as intended, 
yet the plan proves inadequate or inappropriate for the situation. They typically represent poor 
judgment, improper choice of procedure, or the misinterpretation or misuse of relevant 
information. Finally, perceptual errors occur when sensory input is degraded or ‘unusual,’ and 
can result in misjudged distances, altitudes, and descent rates, as well as a myriad of visual 
illusions. 


Violations. In contrast to the three error forms described previously (decision, skill-based, 
and perceptual), violations represent a willful departure from those practices deemed necessary 
to safely conduct operations, and as such are differentiated from errors. Violations are further 
divided into two types based upon the characteristics of individuals committing them and those 
who govern their actions. Routine violations tend to be habitual by nature and are typical of the 
individual's behavioral repertoire. Equally important, routine violations are often perpetuated by 
a system of supervision and management that tolerates such departures. Exceptional violations, 
on the other hand, are isolated departures from authority, neither typical of the individual nor 
condoned by management. 
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Table 1 

Selected examples of Unsafe Acts of Pilot Operators (Note: This is not a complete listing) 


Errors 

Skill -based Errors 

Breakdown in visual scan 
Failed to prioritize attention 
Inadvertent use of flight controls 
Omitted step in procedure 
Omitted checklist item 
Poor technique 
Over-controlled the aircraft 

Decision Errors 

Improper procedure 
Misdiagnosed emergency 
Wrong response to emergency 
Exceeded ability 
Inappropriate maneuver 

Perceptual Errors (due to) 

Misj udged distance/altitude/airspeed 
Spatial disorientation 
Visual illusion 


Violations 

Failed to adhere to brief 
Failed to use the radar altimeter 
Flew an unauthorized approach 
Violated training rules 
Flew an overaggressive maneuver 
Failed to properly prepare for the flight 
Briefed unauthorized flight 
Not current/qualified for the mission 
Intentionally exceeded the limits of the 
aircraft 

Continued low-altitude flight in VMC 
Unauthorized low-altitude canyon 
running 
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Linking Technologies to Human Error 

Figure 12 shows the percentages of the AvSP products that target each type of unsafe 
act. Of the products that address unsafe acts, most target decision errors, by providing 
better information, more highly automated systems that eliminate or reduce the need for 
certain decisions, or through training. Fifteen percent of the products address all errors in 
general, and only one of the products primarily targets skill-based errors. None of the 
products solely targets perceptual errors, albeit the Synthetic Vision Display products 
address perceptual errors but are ubiquitous, targeting decision and skill-based errors as 
well. None of the products primarily targets violations. Nearly half of the products did 
not directly target specific unsafe acts at all. 

100 - 

90 - 


w 80 - 
o 



Decision Skill-Based Perceptual All Errors Routine Exceptional N/A 

Errors Errors Errors Violations Violations 


HFACS Unsafe Acts Category 

Figure 12. Percentages of the AvSP products that address the Unsafe Acts subcategories 
of the Human Factors Analysis and Classification System (HFACS). 

Figures 13 shows the percentages of the AvSP products that target each of the unsafe 
acts subcategories of HFACS, broken down by each cell of the Haddon Matrix. The 
products that focus on human-related, pre-event factors nearly evenly target decision 
errors, skill-based errors, and all errors. The products that primarily address vehicle- 
related pre-event factors are divided between those targeting decision errors and those 
that do not address any unsafe acts, with a small percentage targeting all errors. Most of 
the products focusing on environment-related, pre-event factors do not address any 
unsafe acts, but one targets decision errors and one targets all errors. Half of the products 
primarily addressing vehicle-related, during-event factors target decision errors or all 
errors, and the other half do not address any unsafe acts. The single product that focuses 
primarily on vehicle-related, post-event factors does not address any unsafe acts. 


15 


HUMAN 


System 

VEHICLE 


ENVIRONMENT 


PRE-EVENT 


a> 

o 

c 

a> 

D 

O' 

a> 

CO 


EVENT 


POST-EVENT 



DE= Decision 
Errors 

SBE= Skill-Based 
Errors 

AE= All Errors 

Percentage of 
AvSP Products 

P 60-69.9 
P 50-59.9 
J40-49.9 
j 30-39.9 
j 20-29.9 
j 10-19.9 
|0.1-9.9 
|0 


Figure 13. Haddon Matrix, with embedded graphs indicating the percentages of all of the 
AvSP products that target each HFACS Unsafe Acts category in each cell. 
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Evaluating Intervention Effectiveness 

Evaluating the effectiveness of an individual safety intervention or group of 
intervention strategies involves two major issues. These are (1) the extent to which the 
interventions actually target the problems or hazards that are adversely affecting safety, 
and (2) the extent to which interventions will actually affect or reduce the hazard that it 
targets. The first issues can best be determined by examining the repository of actual 
accidents and incidents to determine what has caused accidents in the past. The second 
involves estimates of impact via a variety of methods. We will address each of these 
issues in turn in the following sections. 

Appropriateness of Intervention Focus 

In a series of studies, Shappell and Wiegmann (Shappell & Wiegmann, 2001, 2002; 
Wiegmann & Shappell, 2001, 2002) have analyzed the human causal factors associated 
with U.S. commercial and general aviation accidents using HFACS. The results are 
summarized in Figure 14. The top panel represents the percentage of commercial 
accidents (FAR Part 121 and 135 scheduled and non-scheduled air carriers) that were 
attributed to an unsafe act by the aircrew and the bottom panel represents general aviation 
(FAR Part 91) accidents. As can be seen from the figure, approximately one third of all 
commercial and general aviation accidents are due to decision errors by pilots and very 
few are due to perceptual errors. Therefore, the finding in the previous section noting 
that the majority of AvSP interventions target decision-making, but few address 
perceptual errors, is reasonable and prudent. 




General Aviation 


Skill-based Decision Perceptual Violations 

Errors Errors Errors 


Figure 14. Percentage of commercial and general aviation accidents associated with 
various unsafe acts by pilots. 
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Also indicated in Figure 14, however, is that skill-based errors are the primary unsafe 
act causing accidents, and that violations of the rules also contribute to approximately one 
quarter of these occurrences. Yet, few AvSP interventions directly target skill-based 
errors and none focus primarily on reducing violations. Therefore, from this analysis, it 
appears that there may be gaps in the intervention being examined by AvSP and that a 
broader group of interventions may be needed. 

Intervention Impact 

To assess the potential impact that an intervention might have on reducing a hazard or 
particular unsafe act (i.e., human error) is more difficult than determining its applicability 
or relevance given the actual causes of accidents. Such estimates of impact are generally 
derived from expert judgments, such as those used in the generation of Bayesian belief 
networks (Renooij, 2001). Such judgments, however, should be based on information 
available from the literature describing the efficacy of the intervention under controlled 
conditions or effectiveness of applications of the intervention in other domains (Runyan, 
1998). They could also be based on information about similar types of interventions 
associated with other problems or related dimensions of the intervention. To accomplish 
this, however, a repository or database containing relevant literature on interventions 
needs to exist, and more importantly, the manner for searching and retrieving such 
literature needs to be compatible with the goal of evaluating intervention strategies. 

To accomplish this objective we have develped a prototype database and search 
engine for retrieving pertinent human factors literature that could be used to help 
determine the impact that certain technologies might have on specific types of human 
error. As human error is central to our approach, and human error can be understood as 
synonymous to human performance (i.e., poor performance or failure to perform), a 
human performance model is a necessary starting point for the framework mapping errors 
and technologies. Human performance, however, seldom happens in isolation, but is 
affected by a myriad of factors. These factors must necessarily be considered in the 
framework. Also, to successfully map human errors to various technologies and vice 
versa, it is obvious that a set of commonalities between each must be identified. An 
element common to both humans and technologies is the task. Humans use technology 
as tools to accomplish certain tasks, or technologies may require humans to perform tasks 
on them (e.g., maintenance). Hence, a system approach is deemed as the only reasonable 
and useful way of linking human errors with technologies. In essence, the human and the 
machine form a system (Rouse, 1980). According to this definition, "machine" is other 
than human (i.e., technology), and the human-machine interaction can be depicted in an 
input-output diagram, where the human's outputs are the machine's inputs and the 
machine's outputs are the human's inputs (Rouse, 1980; see Figure 15). 
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Figure 15. Bailey's (1982) qualitative model of human performance. 

It is clear that the models must contain at least three critical elements: The human 
operator, his or her task, and the environment or context in which the task is performed. 
There are also several task taxonomies that entail all or parts of these elements, for 
example Gawron, Drury, Czaja, and Wilkins' (1989) human’s taxonomy, which had three 
major branches: Environment, subject, and task. Although the majority of these factors 
remain unknown at worst and poorly understood at best, and although the number of 
variables and their potential interactions can be bewildering, the systems approach we 
have adopted offers some startling benefits in linking such disparate realms as human 
performance and technological innovations. 

Given the multidimensional nature of the problem, the framework adopted for this 
proposition is five-dimensional. It consists of three axes, one for the human operator, one 
for the task, and one for the environment. There are several existing, published 
taxonomies for each axis, too. Because human errors can be mapped to all of these 
dimensions, albeit not uniquely, the fourth dimension of the problem can thus be placed 
within the "molecules" (i.e., three-dimensional cells) in the matrix. Furthermore, 
technologies can be mapped to tasks and environments as well as to human 
characteristics (e.g., visual displays, auditory alarms) and placed within the 
aforementioned framework. Hence, human errors and technologies will co-habit 
molecules in the matrix, linking them together (Figure 16). 
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Figure 1 6. The five-dimensional matrix for mapping human errors and technological 
innovations. 


Initially, we have used combinations of existing human, task, and environmental 
taxonomies for the three axis of the matrix. These taxonomies must remain dynamic, 
however, with new classes, subclasses, and sub-subclasses added or deleted according to 
the known technologies, their applications, and human error types that are entered into 
the framework. Hence, our approach serves as a vehicle for taxonomic development in 
all three areas pertaining to human performance and its validation will be a continuous 
process in lieu of a one-time effort. This framework will also direct further work deemed 
essential for the comprehensive solution to the problem. These efforts will be described 
next. 

To make the proposed framework usable, it must be brought into a database format 
which is flexible, expandable, fully accessible by a multitude of researchers and users, 
and which can be continually developed as new information becomes available. Given 
the magnitude of the project as well as its potential value to the worldwide human factors 
community, a distributed development environment must be considered indispensable. 
We created a prototype database that accommodates all of these requirements and 
facilitates future development effort beyond the present project. The creation of this 
prototype database will be described in detail next. We will discuss the back-end 
(database), front-end (interface), hosting issues, and implementing the initial data set. 

Selection of Database Axis Taxonomies 

We selected the Human Factors Analysis and Classification System (HFACS) as the 
error taxonomy, based on its use by NASA and previous reviews of the human-error 
literature (Shappell & Wiegmann, 2001). HFACS is based on Reason’s (1990) model of 
latent and active failures. As mentioned previously, the framework addresses the errors 
committed by operators as well as the preconditions and organizational factors that 
contributed to such errors. 
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The remaining axis taxonomies were chosen based on the literature review performed 
during Year 1 efforts. Of the task-descriptive taxonomies reviewed, we chose the 
Berliner, Angell, and Shearer’s (1964) as the base taxonomy for the task axis. This 
taxonomy classified tasks according to human information processing characteristics 
(e.g., perception, problem-solving, and decision-making) and descriptive verbs (e.g., 
detects, inspects, interpolates, etc.). Hence, this taxonomy is also compatible with the 
taxonomy on the human axis. 

The taxonomy selected for the human axis is based on the human information 
processing framework, which in turn is based on number of similar models proposed by 
different investigators (e.g., Broadbent, 1958; Smith, 1968; Sternberg, 1969; Welford, 
1976). The framework chosen for our initial application is a composite of these 
(Wickens, 1984) and depicts human information processing in the form of sequential 
stages. Stimuli must be sensed in the first stage and perceived in the second. The third 
stage involves higher cognitive processes, such as decision-making, problem-solving, and 
response selection, and the fourth stage action execution. There are three other 
components in the model that interact with these stages: Attentional resources, and 
working- and long-term memory. Finally, a feedback loop closes the system. 

A special subclass of the human factors taxonomies reviewed is taxonomies of 
environmental factors. Human performance never occurs in isolation and therefore 
taxonomy of human factors cannot be complete without attention to the environment. 
Chambers' (1969) provided a detailed and comprehensive example of the environmental 
variables of consideration. We expanded this taxonomy with another major subclass, that 
of task environment. 

Finally, a technology axis was generated using the list of 48 technologies provided by 
NASA’s AvSP. NASA initially classified the technologies into seven broad categories. 
These categories are accident mitigation, aviation system modeling and monitoring 
system, single aircraft accident prevention, synthetic vision systems, system-wide 
accident prevention, weather accident prevention, and aircraft icing. However, we are 
researching the use of the various technology taxonomies reviewed early (e.g., 
functional-approach) to develop methods for classifying these technology interventions to 
groups based on their theoretical impact. 
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Database 

We opted to create the prototype database as a “weblication” (see Wroblewski & Rantanen, 
2001), anticipating future development work that would be conducted in a widely distributed 
manner. Several practical questions had to be addressed and problems solved to arrive at a 
functional and usable database. The key decisions will be described next. 

Database Software 

A MySQL database was chosen as a back-end for the system for several reasons. Not only is 
MySQL an efficient, full-scale relational database, it is both free and open source and can be 
used with most platforms. Designing a workable table structure was not difficult using a 
relational database. However, because of the hierarchical structure of possible taxonomies it 
might be interesting to investigate using an object-oriented database (or a hybrid structure) to 
store and manipulate the taxonomic structures. This could be particularly useful when moving or 
changing the taxonomic structure. 

Database Structure 

The database structure consists of tables to hold articles, tables to hold taxonomy, including 
description as well as keyword, and tables that match articles to keywords. Figure 17 depicts this 
structure. 

Interface: General Ideas 

Efforts to design of an interface to an information base can generally be divided into two 
parts. The first might be termed "Interface Design,” which describes what the user will 
encounter. This includes the flow of forms and printouts, the number of buttons, labels and the 
type of choices at each juncture. The second part concerns turning this design into reality. This 
includes decisions about the software for producing the forms, retrieving data from and sending 
information to the database and, in a web environment, how to store information about the users 
query during a session. This part might be called "Interface Application". Specific to the latter 
situation are questions about how to give the user access to the organization of the information. 

In this case the articles are being fitted into a taxonomic structure, which presents its own set of 
problems that are addressed in the next section. 
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Figure 17: Database structure. 

Interface Design 

The project is centered around the process of indexing journal articles by fitting them into a 
taxonomic hierarchy. This taxonomy has been described as a 5-dimensional information space. 
However it might also be viewed as a tree-like structure (Figure 18). The problem of how to 
give the user access to the articles in this tree is both technical as well as social. The taxonomy 
is difficult to show all at once and the larger it gets, the more difficult it will be. Not only is this 
difficult to represent on a computer screen, it could overwhelm the user with too many choices. 
Furthermore, the user should not have to click through many screens before retrieving results. 

For this reason, we chose as a guiding principal to have the user start with a simple query and 
then give him or her the ability to make it more complex as desired. Initially the user can 
choose up to five search terms and retrieve results. From the results page, changes can be made 
to the search terms, either widening the net or tightening it, each time displaying the total number 
of hits and the titles. To start with, we concentrated on giving the user access to the information 
only through the taxonomy. Later we will add the ability to search by other metadata such as 
author and journal. The site layout is depicted in Figure 19. 

Future Enhancements 

As this is a prototype database, a number of enhancements to its usability can already be 
identified. We would like to provide context-sensitive help, the ability to download 
bibliographic entries or lists in a variety of formats (e.g., xml, bibTeX and plain text), and add 
thesaurus functionality to it. The latter, in addition to being an added benefit to searching the 
database, could be a useful recourse on its own for normalizing some of the vocabulary in the 
field of study. 
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Figure 18: A tree-like depiction of the database structure 


Web Access to DB 

It is clear that a database of this type will ultimately serve a geographically dispersed group 
of users and will have to be online to be effective. For this reason, we decided that the prototype 
should also be made available online. By creating a web-accessible prototype, various design 
and technical issues such as the ones outlined in this document can be identified. Furthermore, 
by having the prototype database centrally accessible, we can seek assistance from others in the 
field during the taxonomy-building phase. 

Interface Software 

Perl was the programming language of choice for the web application scripts, primarily 
because it is freely available and ubiquitous. Perl also has well-developed libraries and drivers 
that allow connection to the MySQL database. However, on retrospect, perhaps an embedded 
scripting language such as PHP might have also been a good choice. This might have allowed 
both a programmer and web designers to work more closely together to create the interface. 
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Figure 19: A flowchart of database search. 
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Query Development 

In a taxonomy, if a keyword has “children” that belong to it, any characteristics of that 
keyword would also be inherited by its “children”. By design, any query for a keyword should 
also retrieve all items below it in the hierarchy (Figure 20). To achieve this, each keyword in the 
taxonomy is assigned an integer value. For the purposes of querying, it is useful to be able to 
express a search term or keyword along with its place in the taxonomy in one string of 
characters. To do this, we used a series of integers separated by a symbol. The first number 
represents the axis; the next represents its position in the next level and so on. In this case we 
used a colon to concatenate the values together. In order to retrieve the articles that match a 
certain keyword, a similar string of numbers and symbols is used to query matching articles. 
When building the query the fields are concatenated using the same symbol. Finally, in order to 
get all the “children” below the node a string comparison function is used along with simple 
pattern matching, (see: [nasa-heti.org/docs/diagrams/query.gif]) 



HUMAN FACTORS/AVIATION 


AXIS 


Technology 


Error 


Human 


A query for this... 


Should return everything 
below it 


Figure 20: A keyword and its “children” in a database query. 

In the relational database model, building the sort of queries we need to retrieve multiple 
record sets from one table is typically done using subqueries. However, MySQL does not yet 
support subqueries. In order to work around this issue, we used an alias for the table for each 
subset we need (Figure 21). 
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One particular query design issue remains for the future. During the taxonomy-building 
phase, initial query choices draw from existing keywords. In other words, users can choose from 
keywords that have been assigned to other articles. Presumably, after a certain amount of time, 
the taxonomy will be rebuilt. Keywords that are synonyms will be consolidated, groupings will 
be established and new terms created to "flush out" certain categories. This is important for two 
reasons. One of the goals of this project was to develop complete taxonomies for areas in this 
field. Another is to determine what research is being done or not being done within in these 
areas. Hence, the taxonomy should reflect all possible keywords, not just what has already been 
studied. Although not technically difficult, it is important that after the building phase is over 
the query choices should come from the taxonomy structure tables instead of existing 
article/keyword table. 
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Figure 21: Subquery structure. 

Website Maintenance 

An additional problem in a web environment is keeping track of user information while he or 
she moves through the site. We have implemented server-side sessions to keep track the user's 
editing privileges and most recent query terms. We have also implemented a layered permission 
system for users with five levels of permission: 


read only — search database 

all of above + add articles, choose from existing keywords 

all of above + add keywords entries to taxonomy 

all of above + modify/delete articles entries 

all of above + modify/delete keyword entries in taxonomy 
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Hosting Issues 

A commercial Internet provider hosted the prototype database during the initial development. 
It was moved to www.humanfactors.uiuc.edu for a permanent home and to allow for further 
development and maintenance of the database 

Initial Data Set 

We used an initial data set consisting of metadata from approximately 400 articles from the 
International Journal Aviation Psychology. The articles have been indexed according to index 
terms given by the journal itself. These index terms may be one keyword or may also have a 
subheading. Instead of creating a new set of keywords, we fitted these existing keywords into 
the taxonomy. In some cases this was easy. In others cases, making the keywords fit into the 
new taxonomy was problematic. For example, "displays»attitude (ADI)" clearly fits into the 
technology axis. Thus "displays" becomes the domain and "attitude (ADI)" becomes a category. 
However "air traffic control»selection" could either be in environment or in task depending on 
the article (Figure 22). 
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Figure 22: Fitting existing index terms from the International Journal of Aviation Psychology 
into the database taxonomy 
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Discussion 


Proper classification of technologies is of critical importance to the outputs of our proposed 
framework and the prototype database, that is, accurate and comprehensive linking of 
technologies with human error. However, this task can be successfully completed only by 
conducting thorough task analysis of every technology in every application and in every 
environment and by all potential users. Unfortunately, there exists very few published task 
analyses, even for existing technologies. Although this lack of obligatory data for out framework 
can be seen as a drawback of the approach we adopted, it is clear that detailed scrutiny of the 
entries in the framework (i.e., technologies) is imperative if any useful information is to be 
gained. Consequently, a broad, general approach in the domain of human-machine interaction 
where innumerable variables and their interactions exist simply will not be justifiable. Hence, 
the progress that is achievable in further development of the framework and its usability are 
inextricably linked to the availability of task analyses associated with various technologies. On 
the other hand, it is conceivable that the proposed framework will become a useful and usable 
database that effectively reduces parallel and overlapping research efforts. 

The scope of the task of providing a comprehensive mapping between technologies and 
human error is only beginning to emerge as the prototype database was developed. However, we 
strongly feel that careful attention to the creation of a robust conceptual framework is essential 
for success of future research, which must proceed with deliberate and systematical manner as 
well. Two main thrusts for further work can be identified. First, to build upon the conceptual 
framework presented in this paper, literature search must be extended to selected domains and all 
available research reviewed. Second, the framework must be "filled" with case studies, which 
will serve as evaluation tools as well as set an example for the complete structure. 

Task analysis is an essential component of our model. Unless a thorough task analysis is 
conducted for each example of technology, we will not know who are the users, how the user(s) 
will use the technology, and how the users' performance will be affected by the technology. The 
method used to answer these questions is task analysis. Several techniques for task analysis exist 
and task analysis is an essential part of any system development. 

During the database-building phase, initial query choices draw from existing keywords. 
Presumably, after a certain amount of time, the taxonomy will be rebuilt based on user-submitted 
keywords. This is important for two reasons. One of the goals of this project was to develop 
complete taxonomies for areas in this field. Another is to determine what research is being done 
or not being done within in these areas. Hence, the taxonomy should reflect all possible 
keywords, not just what has already been studied. Although not technically difficult, it is 
important that after the building phase is over the query choices should come from the taxonomy 
structure tables instead of existing article/keyword table. 

A number of enhancements to the usability of the database can already be identified. We 
would like to provide context-sensitive help, the ability to download bibliographic 
entries or lists in a variety of formats (e.g., xml, bibTeX and plain text), and add thesaurus 
functionality to it. The latter, in addition to being an added benefit to searching the database, 
could be a useful recourse on its own for normalizing some of the vocabulary in the field of 
study. 
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In summary, the proposed database framework will allow for a directed literature search 
and review of both empirical and theoretical research that will help in establishing the direction 
of the impact of a given technology and its application on human error (i.e., a cause of an error or 
a remedy for an error) as well as the particular mechanisms of such relationships. 


Summary and Conclusion 

One of the main factors in all aviation accidents is human error. The NASA Aviation Safety 
Program (AvSP), therefore, has identified several human-factors safety technologies to address 
this issue. Some technologies directly address human error either by attempting to reduce the 
occurrence of errors or by mitigating the negative consequences of errors. However, new 
technologies and system changes may also introduce new error opportunities or even induce 
different types of errors. Consequently, a thorough understanding of the relationship between 
error classes and technology “fixes” is crucial for the evaluation of intervention strategies 
outlined in the AvSP, so that resources can be effectively directed to maximize the benefit to 
flight safety. The purpose of the present project, therefore, was to examine the repositories of 
human factors data to identify the possible relationship between different error class and 
technology intervention strategies. 

The work described here reflects Year 2 efforts that focused on evaluating the 48 technology 
interventions previously generated by AvSP. Our evaluation consisted of several steps. The first 
step was to analyze each intervention based on its underlying nature or function as an 
intervention. This analysis involved clustering interventions into groups or categories using 
various taxonomies of intervention strategies. The goal of this effort was to determine the 
general focus or distribution of interventions. The second step involved mapping these 
interventions onto error categories to determine the extent to which the interventions addressed 
various classes of human error across different stages of accident causation (e.g., pre-crash, 
crash, and post-crash events). Finally, the third phased focus on the issue of evaluating the 
magnitude of impact that each intervention might have on the actual error category it targeted. 

The results of this work suggest that AvSP technologies primarily address decision making 
and decision errors and that very few if any specifically target skill-based or perceptual errors or 
violations of the rules. The impact that these technologies will have on reducing the accident 
rate, therefore, may be limited given their restrictive scope. In addition, the particular impact that 
each intervention technology may have in reducing decision errors also needs to be examined. A 
prototype database for mapping interventions onto human errors to address this issue was 
proposed. Proper classification of technologies is of critical importance to the outputs of our 
proposed framework and the prototype database, (i.e., accurate and comprehensive linking of 
technologies with human error). Further research is needed to develop methods for using this 
database system to help generate judgments concerning an intervention’s impact so that 
interventions can be effectively implemented and improvements in aviation safety can be 
ultimately realized. 
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AvSP PRODUCT DICTIONARY 


Based on FY03 Project Plan Data 


TAB# 

PRODUCT NAME 

EESSraBB 

PRODUCT FORM 




AM-1 

Next-Generation Crash Analysis 
Codes 

New complete suite of dynamic crash analysis 
software 

Development of advanced analysis modules 
(e.g. Finite Element Modeling (FEM) elements 
& Subroutines) in a functionally integrated 
(e.g. seats and airframe together) suite of 
dynamic crash analysis software to make the 
design process more efficient, integrated, and 
standardized and allow certification of 
crashworthiness as a system rather than by 
individual component testing. 

Aircraft manufacturers, certifiers, and 
operators 

Lack of a functionally-integrated tool 
suite; time and expense of 
individual/empirical design and 
certification analyses and full-scale 
tests 


Reduce 

crashworthiness design 
cycle time by 50%, 
reduce crashworthiness 
validation systems 
testing by 50% 

AM -2 

Energy Absorbing Seats, 
Restraints and Structures 

Designs concepts and test results 

Development of designs for improved energy 
absorbing seats, restraints, and energy 
absorbing aircraft structures to improve 
survivability in accidents. 

Aircraft component manufacturers and 
aircraft manufacturers for small, large 
and transport category aircraft (initial 
focus on general aviation) 

Improve survivability in survivable 
accidents by minimizing loads and 
maintaining habitable volumes. 


Reduce impact loads 
by 50% 

AM -4 

Next-Generation Crashworthiness 
Design Guidelines 

Handbook of materials test results, design and 
injury criteria and structural design concepts 

Development of a handbook containing test 
results of materials tested and design 
guidelines for their use. 

Aircraft seat manufacturers, airframe 
manufacturers and general aviation 
and air carrier operators 

Reduce design/development time for 
new products; improve survivability in 
survivable accidents by minimizing 
toads and maintaining habitable 
volumes 


Reduce 

crashworthiness 
validation systems by 
50% 

AM-5 

Fuel Tank Fire Prevention and 
Fire Suppression System 
Technologies 

Design and test results for fuel tank inerting, cargo 
compartment suppression and breathing oxygen 
generation prototype system, either combined or 
separately 

Development of a fuel tank inerting, cargo 
compartment suppression and breathing 
oxygen generation requirements definition, 
prototype system and test results. 

Inerting system vendors and regulators 
and air carrier operators 

Fuel tank explosions and halon 
replacement 


Reduce fuel tank 
flammability by 50% 

AM -6 

Cargo Hold Fire Detection and 
Detection Design Guidelines 

Prototype fire detectors (i.e. gas sensors), 
detection system design criteria, test results, and a 
model for cargo hold fire propagation 

Development of prototype fire detectors that 
sense gas products of fire combustion, sensor 
tests and test results, sample fire tests and 
test results, design concept that minimize false 
alarms, plus a model for fire propagation; all of 
which can be used to eliminate high false- 
alarm-rate sensor. 

Detection system vendors, aircraft 
manufacturers and air carrier operators 

High false alarm rate of present 
sensors; operations' cost impacts 


Reduce false-alarm 
rate by 80% 

AM-7 

Elevated Flash Point Fuel 
Technologies 

Data on existing fuels, experimental fuels, and 
prototype fuel additives and test results 

Technology transfer of existing experimental 
fuels, development and prototype fuel 
additives, testing for fire propagation, 
suppression, and flammability reduction and 
documentation of test results. 

Fuel refiners and suppliers, air carrier 
operators and operators using Jet A 
fuels 

Fire propagation in post-crash fire 
environment and lower flammability 
for fuel tank explosions 


Reduce spilled fuel 
hazard by 50% and 
reduce fuel tank 
flammability by 50% 
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Based on FY03 Project Plan Data 


TAB# 

PRODUCT NAME 

PRODUCT FORM 

PRODUCT DEFINITION 

Customer / End User 

TARGETED PROBLEM 

Milestone 

III# 

Pe Tamet nCe 

AVIATION 

SYSTEM MODELING AND MONITORING SYSTEM (ASMM) 

/ 7 



l^el 

ASMM-1 

Incident Reporting Enhancement 
Tools 

Database converted to ORACLE, hardware and 
software to permit electronic report submission and 
test results for analyst decision support system 

Upgrade the 24-year old technology of the 
ASRS database to include: conversion of 
ASRS legacy database to ORACLE; electronic 
submission of reports; and test and evaluation 
of an analyst decision-support system. 

FAA; All personnel in the aviation 
industry including flight crews, flight 
attendants, mechanics, technicians, 
ATC, airport operations and 
researchers in aviation safety and 
human factors 

Improve the collection and utilization 
of ad hoc safety reports from front 
line NAS-wide personnel. 



ASMM-2 

National Aviation System 
Operational Monitoring Service 
(NAOMS) 

Survey tool and validated methodology. 

This element aims at the permanent field 
implementation of a National Aviation 
Operational Monitoring Service (NAOMS) 
responsible for developing and maintaining a 
comprehensive and coherent survey of the 
safety and performance of the NAS from the 
perspective of front line personnel NAS-wide. 
It is a proactive companion to the ad hoc 
submittal process embodied within ASRS. 

FAA; All personnel in the aviation 
industry including flight crews, flight 
attendants, mechanics, technicians, 
ATC, airport operations and 
researchers in aviation safety and 
human factors 

Creation of a mechanism for 
measuring the overall safety of the 
NAS in a quantitative, precise, and 
repeatable way on an ongoing basis 



ASMM-3 

Aviation Performance Measuring 
System (APMS) Tools 

PC-based software, documentation, training, and 
guidelines for APMS utilization 

APMS is an integrated suite of tools to facilitate 
the implementation of routine flight-data 
analyses within each of the air-service 
providers. APMS develops and documents the 
software and procedures for data management 
and analyses of Flight Operational Quality 
Assurance (FOQA) data that enable users to 
easily interpret implications in safety and 
efficiency of flight. 

Air carriers and general aviation 
operators -all functional disciplines, 
including flight operations, training, 
engineering and maintenance 

Enable each user to implement 
proactive management by monitoring 
and analyzing large flight-recorded 
databases on a continuous basis. 



ASMM-4 

Performance Data Analysis & 
Reporting System (PDARS) Tools 

Hardware: PCs, printers, tape drives; Networking: 
Routers, hub, switches, DSL&T1 lines; Taps: to 
TRACONs and Centers; Software: tap client, 
server/client modules, central merge process, site 
adaptation; User materials: Customized sample 
report scripts, help information and manuals, 
training. 

PDARS provides the capability to: collect and 
process Air Traffic Control (ATC) operational 
data (i.e., radar track and Enhanced Traffic 
Management System; ETMS); compute 
quantitative operational performance measures 
on a regular basis relating to system safety, 
delay, flexibility, predictability and user 
accessibility; conduct causal analyses and 
operational problem/issue identification and 
analyses; access operations' design and 
simulation tools for “what-if analyses and for 
identification and evaluation of system 
improvement options; archive performance 
statistics and basic operational data for use in 
research, development, and planning studies. 

FAA 

Continuous, routine monitoring of 
performance metrics to enable the 
implementation of a policy of 
proactive NAS management. 



ASMM-5 

Fast-time Simulation of System- 
wide Risks 

A hierarchy of mathematical models 

Rigorously validated system-wide models and 
simulations of relationships among elements of 
the NAS to support predictions and safety-risk 
assessments of system-wide effects of new 
flight and ATC technologies and/or procedures 
before they are inserted into the operating 
environment. Includes engineering models, 
operating concept models, support/logistics 
models, human performance models and risk 
analyses. 

FAA. NASA and researchers in aviation 
safety and human factors 

Ability to assess the safety impact of 
new technologies before they are 
implemented 
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ASMM-6 

Prototype System-wide Risk 
Assessment Capability 

PC-based software, documentation, training, and 
guidelines for utilization. 

A capability that demonstrates the feasibility 
and value of automatically merging de- 
identified disparate data sources to assess 
system-wide safety risks. 

FAA; all personnel in the aviation 
industry including, flight crews, cabin 
crews, mechanics, technicians, ATC, 
airport operations, and researchers in 
aviation safety and human factors. 

Creation of a capability to (1) provide 
decision makers with reliable 
information on safety of the NAS; (2) 
identify causal factors, accident 
precursors, and off-nominal 
conditions; and (3) enable and 
encourage sharing of information. 


1 ai 
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1RCRAFT ACCIDENT PREVENTION (SAAP) 




warn 

hAhi 

SAAP-1 

Condition-Based Maintenance 
Technologies 

Handbook of guidelines for implementaiton of 
condition-based maintenance 

Selection of an aircraft system component 
(e.g., landing gear), implementation of a 
prototype condition-based maintenance 
program, followed by creation of a handbook 
containing operator guidelines for 
implementation of a condition-based 
maintenance program. 

FAA and aviation operators 

Component failures 

2,6 


SAAP-2 

Real-Time Health Monitoring 
Technologies 

Mathematical models 

System and component mathematical models 
to enable real-time, in-situ diagnostic and 
prognostic health monitoring systems. Initial 
target is propulsion systems - other sub 
systems will be selected later. 

Aircraft subsystem suppliers, airframe 
manufacturers and air carrier operators 

Component failures 

2.6 


SAAP-3 

Off-Nominal Condition Detection 
& Alerting 

Aircraft 'state versus phase-of-flight' hazard lists 
and results of flight simulation tests to determine 
aircrew interactive cueing methodologies 

Development of aircraft configuration hazards 
versus phase of operation data set, running 
flight simulation trials and developing intuitive 
model-based, crew alert and cueing 
methodologies. 

Aircraft and avionics manufacturers 
and air carrier operators 

Human errors in detection and 
management of malfunctions 

2,3 


SAAP-4 

Outside-of-Normal-Flight 
Envelope Aero-model Definition 

Validated process for developing and validating 
large angle/rate mathematical models for control 
upset training; also B757-specific“outside-of- 
normal-flight" mathematical models. 

Development of B757-specific “outside-of- 
normal-flight" envelope aerodynamic-math 
model through wind tunnel testing of a B757 
model and validation of the process. 

Transport category training 
organizations, aircraft simulator 
developers, aircraft manufacturers and 
air carrier operators 

Upset prevention/recovery 

1.3 


SAAP-5 

Guidance and Control 
Countermeasures 

Flight control software algorithms 

Flight control software algorithms to implement 
guidance and control measures for recovery 
from failure/loss-of-control conditions. 

Avionics and aircraft manufacturers 
and air carrier operators 

Upset prevention and recovery 

1.3,6 


SAAP-6 

FAA Advisory Circular and Flight 
Control Systems Verification 
Methods 

Advisory Circular content 

Development of supporting data for an 
advisory circular that implements modern 
methods for real-time flight software 
verification. 

FAA, flight control software suppliers 
and aircraft manufacturers 

Increased fidelity of delivered flight 
software 

5,8 


SAAP-7 

Integrated Flight-Critical and Non- 
Flight-Critical Architectures 

Design guidelines for fault-tolerant integrated flight- 
critical system designs 

This task takes a standard avionics suite with 
integrated flight-critical and non-flight-critical 
functions and induces failures in the non- 
critical functions to assess the impact on flight- 
critical functions. Design methods to prevent 
failure propagation are then developed. 

FAA, avionics manufacturers, and air 
carrier operators 

Futures group accident precursors 

9 


SAAP-8 

Regional Engine Superalloy Disk 

Disk Component 

Development of a full-scale dual 
microstructure heat treated Nickel-based 
supperalloy disk (Deliver test results and disk 
component indicating a disk that has higher 
fatigue resistant and creep growth properties 
better than in-service baselines.) 

Engine manufacturers 

Disk component failures 

6.12 


SAAP-9 

Advanced Engine Containment 
Case 

Engine Containment System 

Develop a full-scale advanced engine 
containment case and new computational 
methods and design simulation tools (Test 
results indicating more impact resistant and 
structurally lighter containment systems 
compared to current existing in-service 
baselines. Results of numerical analysis 
support certification considerations.) 

FAA and engine manufacturers 

Engine containment failures 

2.3 
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SVS-1 

Synthetic Vision Technology for 
Commercial and Business Aircraft 

Demonstration avionics hardware, software, test 
results, design guidelines and certification strategy 

Situational awareness enhancement system 
utilizing database, sensor, and hazard (terrain, 
traffic - surface and airborne, etc.) detection 
technologies merged with display symbology 
and precise GPS navigational information to 
create synthetic views of the aircraft's external 
environment for display to the flight crew. 

Avionics manufacturers and operators 
from general aviation through transport 
category aircraft 

CFIT, approach and landing errors, 
and runway incursions 



SVS-2 

Synthetic Vision Technology for 
General Aviation (GA) Aircraft 

Demonstration avionics hardware, software, test 
results, design guidelines and certification strategy 

Situational awareness enhancement system 
utilizing database with display symbology and 
precise GPS naviational information to create 
synthetic views of the current external 
environment for display to the fliqht crew. 

Avionics manufacturers and operators 
of general aviation aircraft (focus on 
single/multi-engine piston) 

CFIT, approach and landing errors 
and spatial disorientation upset (loss 
of control) 



SVS-4 

World-wide Geospatial Databases 

Example regional geospatial databases 

Example regional databases and integrity 
monitoring technologies to provide (acquire, 
verify, and maintain) worldwide geospatial 
databases suitable for synthetic vision 
applications. 

Avionics manufacturers, aeronautical 
information providers, and operators 
from low-end general aviation through 
transport category 

CFIT, approach and landing errors, 
runway incursions and spatial 
disorientation upset (loss of control) 



SVS-5 

Runway Incursion Prevention 
Technologies 

Runway incursion detection and avoidance 
algorithms 

Utilizes own aircraft and other aircraft or 
vehicle positions to alert the flight crew of 
potential runway incursion events. 

Avionics manufacturers and operators 
of transport category aircraft. 

Runway incursions 
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SYSTEM-WIDE ACCIDENT PREVENTION (SWAP) 





SWAP-1 

Human Performance Models 

Software 

Human performance models that will permit 
the prediction of human errors due to 
procedural non-compliance, high workload, 
poor situational awareness and inadequate 
crew coordination issues. 

In conjunction with the next product - 
used by aviation product 
manufacturers and designers during 
product development 

Focus on reducing device 
susceptibility to human errors 



SWAP-2 

Crew Tracking Activity 

Software 

Model of intent inferencing to determine what 
the crew was attempting to accomplish, what 
procedures are being carried out and where an 
error occurred, or a chain of errors began that 
may have led to a procedural non-compliance. 

In conjunction with product above - 
used by aviation product 
manufacturers and designers during 
product development 

Focus on reducing device 
susceptibility to human errors 



SWAP-3 

Pilot Skill Training for Cockpit 
Automation 

Written guidelines and reports 

Development of improved training methods for 
flight crews that focus on the increased use of. 
and integration of, automation in the cockpit. 

Air carrier training departments and 
flight training schools 

Focus on automation related human 
error 



SWAP-4 

Training Modules and Simulators 
for General Aviation 

PC-based, web-based, or video training modules 

Development of training modules and 
simulators targeted at improving and 
maintaining general aviation and rotorcraft pilot 
skills in areas especially vulnerable to error. 

Pilots and operators of light fixed-wing 
aircraft 

Focus on general aviation- crew 
training and currency 



SWAP-5 

Instructor Training and 
Evaluation 

Written guidelines and reports 

Development of improved instruction in pilot 
training courses and methods to assess 
effectiveness of training. 

Air carrier training departments and 
flight training schools 

Focus on reduction of human error 
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SWAP-6 

Maintenance Risk and Task 
Analysis Tools 

In some cases, electronic versions of the tools will 
be developed, otherwise written guidelines 

The risk and task analysis tools are based on 
existing tools in the nuclear power industry. A 
variety of protocols and processes will be 
developed for maintenance human factors: (a) 
risk analysis of procedures to determine 
appropriate levels of inspection and to 
streamline inspection and engineering 
processes; (b) task/risk analysis of procedures 
to incorporate human factors principles of 
situational awareness, team coordination, 
communication and resource management; 
and (c) task analysis of procedures for 
improving usability and document structure. 

Air carrier maintenance departments 
and NASA 

Focus on risk reduction 



SWAP-7 

Maintenance Resource 
Management (MRM) Training 
Program for Maintenance 

Recommendations, guidelines and lessons learned 

Focus is development of recommendations 
and guidelines to assist operators in the 
implementation of MRM principles into their 
organization. In addition to recommendations 
and guidelines, assessment tools are 
developed for the purposes of evaluating MRM 
skills and training. These tools cover a wide 
range from individual attitude surveys (applied 
both pre and post training), interview protocols 
and models for developing Return-on- 
Investment (ROI) metrics. New assessment 
techniques incorporating behavioral markers 
are also being developed and tested. 

Air carrier maintenance, training or 
safety departments and contract 
maintenance operators 

Focus on reduction of maintenance 
errors 



SWAP-8 

Augmented/Virtual Reality 
Displays 

Hardware/software and test results 

a) Image-based communication and 
advisement system which includes the 
equipment and human processes involved in 
an image-based communication system that 
enables collaborative problem solving, 
advisement, and documentation. Imagery in 
CEST is comprised of scenes and views of 
workpieces and objects that are referents in 
the problem solving and advisement 
communications, and is recordable. 

Air carrier maintenance departments, 
contract maintenance facilities, FAA 
and general aviation fleet operators 

Focus on reduction of maintenance 
and inspection errors 






b) Development of a virtual-reality device to 
train maintenance inspectors and test results. 
Initial prototype will be a CBT (computer-based 
training) interactive training tool to augment 
existing classroom and on-the-job inspector 
training. This product is based on existing 
virtual-reality hardware that will be upgraded 
and appropriate software developed. 





SWAP-9 

Human Factors Tools 

Recommendations, guidance, informative white 
papers, and research results 

Human factors knowledge-sharing including 
issues and priority checklists for Project HF 
needs, cross-Project integration issues, display 
intuitiveness checklist, Program relevant 
bibliography 

Manufacturers, developers and AvSP 
Projects developing human-centered 
products 

Focus on human friendly product 
design 




Updated: March 26, 2003 by M. Reveley and S.M. Jones 


Page 7 of 11 
Version 4.0 


Based on FY03 Project Plan Data 


TAB# 


PRODUCT NAME 


PRODUCT FORM 


PRODUCT DEFINITION 


Customer /End User 


TARGETED PROBLEM 


WxAP-1 

Cockpit Weather System 
Technologies for Enhanced 
Situational Awareness & Decision 
Making 

Aviation weather information system guidelines for 
transports and general aviation aircraft 

Results to support certification considerations, 
feasibility of implementation, and operational 
benefits of technologies including condition 
based maintenance 

Pilots, weather service providers, 
avionics manufacturers 

Inability of pilots to understand and 
effectively use cockpit weather 
information 



WxAP-2 

Airborne Weather Reporting 
Sensor Technologies 

Sensor design guidelines and test results. 

Complete development of sensor design 
guidelines and provide associated test results. 
Flight demonstrate the dissemination of 
atmospheric data collected by the airborne Wx 
reporting sensor system, from a GA aircraft to 
a Wx product provider, in an acceptable data 
format 

FAA or other weather service providers, 
NAS-wide air and ground users and 
non-aviation users (larger audience 
than aviation) 

Lack of sufficient atmospheric data 
collection in the lower atmosphere 
that is needed for improved aviation 
forecasts and operational updates 



WxAP-3 

Weather Information Datalink 
Systems Technologies for 
Ground-to-Air Dissemination 

Datalink system design guidelines; datalink 
architecture guidelines 

Complete development of datalink 
communication system design and architecture 
guidelines. Flight demonstrate via the 
dissemination of weather products to transport 
aircraft cockpit in national and international 
environment, and to GA aircraft cockpit in a 
national environment 

Datalink service providers, avionics 
[manufacturers and NAS-wide operators 

Poor dissemination of weather 
information to the flight deck 



WxAP-4 

Airborne Weather Reporting 
Datalink Systems for Air-to- 
Ground and Air-to-Air 
Dissemination 

Datalink system design guidelines; airborne 
weather datalink architecture guidelines 

Complete development of datalink 
communication system design and architecture 
guidelines for airborne Wx reporting. Flight 
demonstrate the dissemination of atmospheric 
data from a GA aircraft to a Wx product 
provider and the dissemination of turbulence in 
situ data from one aircraft to another aircraft as 
well as to a weather product provider 



Datalink service providers, avionics 
manufacturers and NAS-wide operators 

Lack of airborne-originated data 
dissemination capability at lower 
altitude operations 




| Milestone j 
Lll# 


Performance 
Target 
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WxAP-5 

Turbulence Characterization 
Technologies 

Validated turbulence models; in-situ algorithms 

Complete development and validate 
turbulence models and in-situ algorithms. 
Flight demonstrate the dissemination of in-situ 
turbulence data from one support aircraft to 
another support aircraft resulting in display of 
transmitted turbulence hazard information in 
receiving aircraft cockpit; flight demo 
dissemination of in-situ turbulence data from 
support aircraft to a Wx product provider in an 
acceptable format 

Pilots, FAA and other weather service 
providers, air carrier operators and NAS 
wide air and ground users 

Lack of atmospheric models for 
support of turbulence technology 
development 

— Lll ¥ — _ 


WxAP-6 

Forward-looking Turbulence 
Sensor Technologies 

Algorithms and test results, radar/lidar prototype 
sensors and test results 

Complete development of algorithms and 
provide test results; complete development of 
radar/lidar prototype sensors and provide test 
results. Flight demonstrate radar and lidar 
warning systems in convective and clear-air 
atmospheric conditions, respectively. 

Pilots, Avionics and radar 
manufacturers, air carrier operators 
and turbo prop/jet operatoions 

Lack of adequate or timely 
information in the cockpit leading to 
avoidance of in-flight turbulence 
encounters 



WxAP-7 

Turbulence Warning & Alerting 
Technologies 

Cockpit alerting guidelines; feasibility study results 
of the use of existing controls to mitigate 
turbulence encounters; turbulence hazard metric 
definitions 

Complete development of cockpit alerting 
guidelines. Provide feasibility study results 
regarding use of existing controls for mitigation 
of turbulence encounters. Complete 
turbulence hazard metric definitions. Flight 
demonstrate turbulence hazard metrics and 
real-time turbulence hazard display in 
convective and clear-air atmospheric 
conditions, as appropriate. 

Pilots, FAA and other weather service 
providers, air carrier operators and NAS 
wide air and ground users 

Lack of consistent turbulence 
alerting/warning methods, mitigation 
of injuries due to transport in-flight 
turbulence encounters 
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Al-1 

Icing Computational Tools 

Computational simulation software 

Validated computational simulation software 
allowing for the examination of ice formation, 
ice protection system behavior, and the 
aerodynamic consequences of ice deposition. 

Aircraft manufacturers, original 
equipment manufacturers, and 
regulatory authorities 

Lack of validated icing simulation 
software 

4,6 8 


AI-2 

Icing Experimental Methods 

Experimental methods, guidelines, and manuals 

Experimental methods allowing for the 
examination of ice formation, ice protection 
system behavior, and the aerodynamic 
consequences of ice deposition. 

Aircraft manufacturers, original 
equipment manufacturers, and 
regulatory authorities 

Lack of sufficient understanding of 
experimental methods with respect to 
in-flight icing 



AI-3 

Icing Experimental Databases 

Experimental Databases 

Publicly available experimental databases 
allowing for the examination of ice formation, 
ice protection system behavior, and the 
aerodynamic consequences of ice deposition. 
Specific topic areas include modern airfoils 
and droplet impingement. 

Aircraft manufacturers, original 
equipment manufacturers, and 
regulatory authorities. 

Lack of comprehensive, publicly 
available databases with respect to in^ 
flight icing. 

3 


AI-4 

Icing Avoidance Technologies 

Completion of field testing of prototype ground- 
based icing remote sensing system and written 
report. 

Prototype ground-based remote-sensing 
system field tested in an airport environment 
that will: (1) improve pitot interpretation and 
management of icing weather hazard 
information transmitted to the cockpit; (2) 
increased all-weather dispatch and better 
management of air traffic in adverse weather 
conditions; (3) reduced in-flight icing incidents 
and accidents during approach and landing; 
and (4) increase all-weather safety and 
capacity 

Pilots, dispatchers, forecasters, and air 
traffic control 

Lack of real time information on icing 
conditions aloft. 

12 7 


AI-5 

Icing Tolerant Aircraft 
Technologies 

Demonstration test through simulation (using icing 
flight test database) of Smart Icing Systems - Ice 
Management System concept design 

Demonstrate through simulation (using icing 
flight test database) a concept design of an Ice 
Management System that, when fully 
implemented, would sense the presence of ice 
accretion on an aircraft, automatically activate 
and manage the ice protection systems, and 
provide the pilot with feedback including the 
effect on measured aircraft performance, 
stability and control. 

Aircraft Manufacturers, Airlines, and 
Pilots. 

Lack of automated systems in aircraft 
to ensure safe operation in the icing 
environment. 
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AI-6 

Icing Atmospheric 
Characterization Technologies 

Icing instrumentation technologies; cloud 
characterization databases 

Understanding of the super-cooled large 
droplet atmospheric environment and 
development of instrumentation to assess the 
super-cooled large droplet environment. 

Regulatory authorities, aircraft 
manufacturers, original equipment 
manufacturers, icing tunnel facilities, 
meteorologists 

Lack of comprehensive 
understanding of natural icing 
conditions; Lack of validated 
instrumentation technologies in the 
super-cooled large droplet regime. 



AI-7 

Icing Education & Training Tools 

Videos. DVDs, web-based media, and computer 
based trainers 

Provide a resource library of educational and 
training materials on in-flight icing that will be 
available to pilots and dispatchers, including 
the international aerospace community. 

Pilots, pilot trade organizations, safety 
foundations, airline operators, 
dispatchers, regulatory authorities 

Lack of education and training 
materials with respect to in-flight 
icing. 

3 
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